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THE FINITENESS PROBLEM FOR AUTOMATON SEMIGROUPS 

IS UNDECIDABLE 

PIERRE GILLIBERT 



Abstract. The finitcncss problem for automaton groups and semigroups has 
been widely studied, several partial positive results are known. However we 
prove that, in the most general case, the problem is undecidable. 

We study the case of automaton semigroups. Given a NW-deterministic 
Wang tile set, we construct a Mealy automaton, such that the plane admit 
a valid Wang tiling if and only if the Mealy automaton generates a finite 
semigroup. The construction is similar to a construction by Kari for proving 
that the nilpotency problem for cellular automata is unsolvable. 

Moreover Kari proves that the tiling of the plane is undecidable for NW- 
deterministic Wang tile set. It follows that the finiteness problem for automa- 
ton semigroup is undecidable. 
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1. Introduction 

iJ^ I Automaton groups, where first introduced by Gluskov in [5]. This family of 

Qs ■ group is a powerful tool to build example or counter-example to various problems 

^SJ ' in group theory. Alesin in j2] constructs a new counter-example to the unbounded 

CN ■ Burnside problem. 

^^ ' Grigorcuk gave in [5] an infinite 2-group G generated by three involutions, giving 

f^ . an other counter-example to the unbounded Burnside problem. Grigorcuk solves 

CO ' the Milnor problem in [8j [7] , proving that G is of intermediate growth (its growth 

is neither polynomial nor exponential). 

Wilson in J16j answer a question by Gromov, constructing an example of group 
with exponential growth but whitout uniform exponential growth. 

An automaton group is generated by the state of a finite Mealy automaton. 
C^ . Therefore it is natural to ask which classical group-theoretical question are decid- 

able. 

For example, the word problem is decidable. There is an algorithm which, given 
an automaton group (or automaton semigroup) and words in the generators, decide 
wether the words represent the same elements. 

On the other hands Sunic and Ventura construct in [14] examples of automaton 
groups in which the conjugacy problem is not solvable. 

We refere to [H Section 7], for a list of several decision problem on automaton 
semigroup. The finiteness problem has been widely stuicd, several partial positive 
results are known. Klimann proves in |12j that the finiteness problem is solvable 
among invertible-reversible Mealy automata with two states (or two letters). 

In this paper we prove that the finiteness problem for automaton semigroup is 
not solvable. Note that several partial positive results are known (cf. [TSIIT]). 
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The proof relies on a construction by Kari in [10]. Kari construct, given a NW- 
deterministic tile set T, a celular automata Ct, such that the plane has valid tiling 
in T if and only if Ct is not nilpotcnt. Kari also prove that the tiling problem for 
NW-deterministic tile set is unsolvable, hence the nilpotcncy problem for cellular 
automata is undecidable. 

Cellular automaton are similar to Mealy automaon, Kari's construction of in 
[To] can be adapted to Mealy automata. Given a NW-deterministic tile set T 
we construct a Mealy automaton At such that the plane has valid tiling in T if 
and only if {At)+ is infinite, hence the finiteness problem for automata group is 
undecidable. 

The problem is still open for automaton groups. Although the methods of Lecerf 
in |13j , the result of Kari and Ollinger in |llj , proving that periodicity is undecidable 
for cellular automata, suggest that the finiteness problem is also undecidable for 
automaton group. 

2. Basic concepts 

Given a set X, we denote by X" the set of all words of length n over X, that is 
the set of all sequences (xi, . . . , a;„) in X. We denote by X'^ the set of all infinite 
sequences {xk)k&i in X. We denote X* the set of all finite words. Given u G X* 
and V G X* U X'^ , we denote by uv the concatenation of the words u and v. We 
also denote 



X* = IJ X" 

X^" = {ueX*\\\iu<n]= U X^ 

k<.n 

X<" ^{ueX*\ Ihw < n} = \JX^ 



Given x G X we denote by a:" the constant sequence of length n which take the 
value X, and by a;" = (x)fcgN the infinite constant sequence. 

A Mealy automaton yi is a 4-tupple {A,J^,6,(j) where A and E arc finite sets, 
S: A X T, ^ A and a: A x T. -^ T, are maps. 

We extend the maps ct: A* x S^" -^ E^" and 5: A^" x E* ^ A^" in the usual 
way. We also denote (Ta{u) — CT(a, m) and (5„(a) = 6{a,u), for all a £ A* and all 
M G E*. The equalities (|2.ip - (|2.4p are satisfied, indeed these equalities define the 
extensions of the maps d and a. 

CTa(ww) = CTa(w)CTi-^(a)(u), for aU o £ A* , It £ E* , and w £ E* U E"^. (2.1) 

S^{ab) = Su{a)S„^^u)ib) , for aU m G E*, a G A*, and b G A* U A". (2.2) 

Cab = CT(, o CTq , for all a,b £ A* . (2.3) 

Suv ~ Sv o 6u , for all m, u G E*. (2.4) 

Note that, given a £ A* , the map CTo preserves the length of each word u G E-", 
moreover if w is a prefix of v, then CTa(w) is a prefix of CTa(w). That is CTq is an 
endomorphism of the tree ct*. 

We denote by {A)+ the subsemigroup of EndE* generated by {cto | a G A}, 
equivalentely {A)+ = {ctq | a G A*}. 
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3. Main 
The following definition is due to Wang }15| . 

Definition 3.1. A Wang tile is a tuple t = {tN,ts,tE,tw), viewed as a square 
with colored edges. A tile set is a finite set of Wang tile. A Wang tiling of a subset 
S of Z^, with a tile set T, is a map i: S* — > T. We say that t is valid if, given 
{x,y) £ Z^, the following equalities hold 

t{x, y)N = t{x, y + l)s, if (x, y) & S and {x, y + l)eS. 

t{x, y)E = t{x + 1, y)w, if {x, y) e S and (x + 1, y) 6 S. 

A simple compactness argument give the following classical result. 

Theorem 3.2. Let T be a tile set. The set 1? has a valid Wang tiling if and only 
if each finite subset of l? has a valid Wang tiling. 

Remark 3.3. In particular, if 1? has no valid Wang tiling, then there is a least 
integer n £ N such that {0,l,...,n}^ has no valid Wang tiling. 

The existence of valid Wang tiling is hard to determined, as illustrate the fol- 
lowing result of R. Berger in [3] . 

Theorem 3.4 (Berger). It is undecidable wether a finite tile set has a valid Wang 
tiling for 1? . 

The following notion was introduced by Kari in |10] . 

Definition 3.5. A tile set T is NW- deterministic if each tile is determined by the 
north and west colors. That is ijv = sn and tw = sw huply that t = s, for all 
s,teT. 



Theorem 13.41 is generalized by Kari in [10] . 

Theorem 3.6 (Kari). It is undecidable wether a finite NW- deterministic tile set 
has a valid Wang tiling for 1? . 

The main goal was to generalize a result of Culik, Pachl, and Yu in [4] to dimen- 
sion one. Kari prove the following theorem in [TOj . 

Theorem 3.7 (Kari). It is undecidable wether a one- dimensional cellular automa- 
ton is nilpotent. 

The argument can be adapted to automaton semigroups, however we need to 
be careful about a side effect (Cellular automata act on words indexed by Z, each 
element of an automaton semigroup acts on words indexed by N. We first define 
a Mealy automaton from a tile set (Kari uses a similar construction to obtain a 
cellular automata). 

Definition 3.8. Let T be a finite NW-deterministic tile set. The Mealy automata 
of T is the tuple At ~ (Q, S, 5, a), where Q = E = T U {-L}, and the maps 6 and 
(T are defined by 

{x,y) (-> y 
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Figure 1. Wang tilings. 

The new state does not depend on the old one, the automaton only remember the 
previous letter. 

{s,t) t-> r 
{s,t) ^ ± 



if rjv = ts and r^ ~ se- 
otherwise. 



That is, given s,t, r G T, if the Wang tiling on the left hand side of Figure [T] is 
valid, then <7{s,t) = r, in all other cases a{s,t) = _L. 

Remark 3.9. The Mealy automaon of a finite NW-deterministic tile set T should be 
understood in the following way. A word m; in Q, can be seen as tiles on the diagonal 
of a square, the Mealy automaton transform this word, to the corresponding tiles 
that can be placed bellow the diagonal. If it is imposible to put a tile at some place, 
then the "mistake" tile _L is placed instead. 



Remark 3.10. Note that Sx{y) = y for all a; G Q and y G S. It follows that: 
aa:{u) = a.j;{uQ){au^{uk+i))keTi, for all u = {uk)keN G S", and all x G Q. (3.1) 

Lemma 3.11. Let T be a finite NW-deterministic tile set. Let t: W? — > T be a 
valid Wang tiling. Consider the word w„ = (i(fc + n, k))k£n for each n G N. The 
equality a^{wn) = l."^Wm+n holds for all n, m G N. In particular all the maps cr™ 
are different. 



Proof. We use the notations of Definition [3^ As the Wang tiling on the right hand 
side of Figure [1] is valid, it follows that 



cTt{^,J){t{i + 1,J + 1)) = t{i + l,j), for ah i,j G N. 



(3.2) 
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Given rt G N, the following equalities hold. 

<y±{wn) = (J±{t{n,0)){a^„+kM){t{n + k + l,k + l)))kefi , by dSH]). 

= ±{t{n + k + l,k))ken, by ([33). 

= ±Wn+l 

The result follows by induction. D 

Lemma 3.12. LetT be a finite NW- deterministic tile set. Ifl? has no valid Wang 
tiling then {At)+ is finite. 

Proof. We use the notations of Definition 13.81 By Theorem 13. 21 there is n £ N such 
that the set {0, 1, ... , n}^ has no valid Wang tiling for T. 

Claim. Let u £ Q^". The following equality holds. 

cr„(xy) = (Ju{x)l-'^ , for all x G S" and all y G S". 

Proof of Claim. We can write u = ui . . . U2n- Set tq = id, and set: 

Tfc = cr„iM2...Ufc = cTufc ° fMfc-i o • • • o cr„^ , for each I <k <2n. 

Notice that 

CTut+i o Tfe = Tfc+i , for ah < fc < 2?!- 1. (3.3) 

Let X G S", let y G S"^. Denote by f{i,j) the (j + I)"' letter of T,{xy), for 
(i,j) G N^ such that i < 2n. That is: 

T,ixy) = /(^, j),gN , for all < i < 2n. (3.4) 

Given < i < 2n, the following equalities hold: 
f{-i + ^,j)]£N^n+i{xy), by (123 

= crui+An{xy)) , by dnSl) 

= ^u,+i(/(«,j)jeN), by dSH) 

= au,^Af{hO)){<Jfi^.J){f{hJ + l)))jeN , by dSI]) in Remark EIO] 
Therefore the following statement holds 

'^f{^J){fihj + 1)) = /(* + 1, J' + 1) , for aU ii,j) G N2 with < i < 2n. (3.5) 

Assume that f{2n, n + k) j^ ± for some fc G N. Applying inductively p.Sp . with 
Definition 13.81 we obtain that /(i + j, i + k) ^ ± for all < i,j < n, and the Wang 
tiling on Figure [2] is vahd. 

Therefore {0, 1, ... , n}^ has a valid Wang tiling; a contradiction. D Claim. 

Let u G Q* be a word of length at least 2n, let v G Q^" and w E Q such that 
u = vw. Let X G S", let y G S'^. We have 

<7uixy) = auwixy) = <7^{a^{xy)) = cr^(cr„(.T)±") = (T„t„(.T)±" = cr„(x)±'^ . 

Therefore {(T„ | m G Q* and Ihw > 2n} is of cardinality at most card(S")(^ •*. 

However {At)+ = {ou \ u G Q^^"} U{(t„ | m G Q* and Ihu > 2n}, therefore the 
following inequality holds 

card(yiT)+ < 1 + cardg + cardQ^ + ■ • ■ + cardQ^"-! + card(I]")(^") . 

Hence {At)+ is finite. D 
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Figure 2. A Wang tiling defined by an element of {At)- 



From Lemma 13.111 and Lemma 13.121 we see that the existence of a valid Wang 
tiling of 1? is equivalent to the infiniteness of an explicit automaton semigroup. 
Therefore, from Theorem 13.61 we deduce the following result. 



Theorem 3.13. It is undecidable wether a given automaton semigroup is finite. 

From the proof of Lemma I3.12( we see the following corollary. 

Corollary 3.14. It is undecidable wether, given an automaton semigroup A and 
/, c Cz A, there exists n such that /" = c. 



References 

[1] A. Akhavi, I. Klimann, S. Lombardy, J. Mairesse, and M. Picantin. On the Finiteness Problem 

for Automaton (Semi)groups, International Journal of Algebra and Computation 22 (2012), 

No. 6. 
[2] S. V. Alesin, Finite automata and the Burnside problem for periodic groups. (Russian), Mat. 

Zametki 11 (1972), 319-328. 
[3] R. Berger, The undecidability of the Domino problem, Mem. Amer. Math. Soc. 66 (1966). 
[4] K. Culik, J. Pachl, and S. Yu, On the limit sets of cellular automata, SIAM J. Comput. 18 

(1989) 831-842. 
[5] V. M. Gluskov, Abstract theory of automata. (Russian), Uspehi Mat. Nauk 16 (1961) no. 5 

(101), 3-62. 



THE FINITENESS PROBLEM FOR AUTOMATON SEMIGROUPS IS UNDECIDABLE 7 

[6] R. I. Grigorcuk, On Burnside's problem on periodic groups. (Russian) Funktsional. Anal, i 

Prilozhen. 14 (1980), no. 1, 53-54. 
[7] R. I. Grigorcuk, On the Milnor problem of group growth. (Russian), Dokl. Akad. Nauk SSSR 

271 (1983), no. 1, 30-33. 
[8] R. I. Grigorcuk, Degrees of growth of finitely generated groups and the theory of invariant 

means. (Russian) Izv. Akad. Nauk SSSR Ser. Mat. 48 (1984), no. 5, 939-985. 
[9] R. I. Grigorcuk, V. V. Nekrashevich, and V. I. Sushchanskii, Automata, Dynamical Systems, 

and Groups, Proc. Stcklov Inst. Math. 231 (2000), 134-214. 
[10] J. Kari, The nilpotency problem of one- dimensional cellular automata, SIAM J. Coniput. 21 

(1992) 571-586. 
[11] J. Kari and N. OUinger, Periodicity and immortality in reversible computing. Mathematical 

foundations of computer science 2008, Lecture Notes in Comput. Sci. 5162 (2008), 419—430, 

Springer, Berlin. 
[12] I. Klimann, The finiteness of a group generated by a 2-letter invertible-reversible Mealy 

automaton is decidable, preprint 2012, 12 pages. 
[13] Y. Lecerf, Logique Mathematique. Machines de Turing reversibles. Recursive insolubilite en 

n G N de I'equation u = 9"u, ou 6 est un "isomorphisme de codes", Comptes Rcndus 

Hebdomadaircs des Seances de L'academie des Sciences, 257 (1963) 2597-2700. 
[14] Z. Sunic and E. Ventura, The conjugacy problem in automaton groups is not solvable, J. 

Algebra 364 (2012), 148-154. 
[15] H. Wang, Proving theorems by pattern recognition-II, Bell System Tech. 40 (1961) 1-42. 
[16] J. S. Wilson, On exponential growth and uniformly exponential growth for groups, Invent. 

Math. 155 (2004), no. 2, 287-303. 

Laboratoire d'Informatique Algorithmique: Fondements et Applications, CNRS UMR 
7089, Universite Paris Diderot - Paris 7, Case 7014, 75205 Paris Cedex 13 
E-mail address: pgillibert9yalioo.fr 
E-mail address: gillibert01iafa.uiiiv-paris-diderot.fr 



